Two modifications of Levenberg-Marquardt's method for fast batch neural network training

نویسندگان

  • Viktor Sobetskyy
  • Stanislaw M. Grzegórski
چکیده

The problems of artificial neural networks learning and their parallelisation are taken up in this article. The article shows comparison of the Levenberg-Marquardt’s method (LMM) and its two modifications JWM (method with Jacobian matrices formed in each step) and BKM (Jacobian calculations only in the first step) for training artificial neural networks. These algorithms have the following properties: 1) simpler calculations; 2) they are partly parallelized. The experiments proved their efficiency. Experimental results demonstrate that neural network for training by them needs a similar number of epochs as the LMM and lesser time for training.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

An Algorithm for Fast Convergence in Training Neural Networks

In this work, two modifications on Levenberg-Marquardt algorithm for feedforward neural networks are studied. One modification is made on performance index, while the other one is on calculating gradient information. The modified algorithm gives a better convergence rate compared to the standard Levenberg-Marquard (LM) method and is less computationally intensive and requires less memory. The p...

متن کامل

Prediction of Groundwater Levels Using Different Artificial Neural Network Architectures and Algorithms

Performance of four types of functionally different artificial neural network (ANN) models, namely Feed forward neural network, Elman type recurrent neural network, Input delay neural network and Radial basis function network and fourteen types of algorithms, namely Batch gradient descent (traingd), Batch gradient descent with momentum (traingdm), Adaptive learning rate (traingda), Adaptive lea...

متن کامل

Numerical and Neural Network Modeling and control of an Aircraft Propeller

In this paper, parametric and numerical model of the DC motor, connected to aircraft propellers are extracted. This model is required for controlling trust and velocity of the propellers, and consequently, an aircraft. As a result, both of torque and speed of the propeller can be controlled simultaneously which increases the kinematic and kinetic performance of the aircraft. Parametric model of...

متن کامل

Implementing the Levenberg-marquardt Algorithm On-line: a Sliding Window Approach with Early Stopping

The Levenberg-Marquardt algorithm is considered as the most effective one for training Artificial Neural Networks but its computational complexity and the difficulty to compute the trust region have made it very difficult to develop a true iterative version to use in on-line training. The algorithm is frequently used for off-line training in batch versions although some attempts have been made ...

متن کامل

Training recurrent network with block-diagonal approximated Levenberg-Marquardt algorithm

In this paper, we propose the block-diagonal matrix to approximate the Hessian matrix in the Levenberg Mar-quardt method in the training of neural networks. Two weight updating strategies, namely asynchronous and synchronous updating methods were investigated. Asyn-chronous method updates weights of one block at a time while synchronous method updates all weights at the same time. Variations of...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • Annales UMCS, Informatica

دوره 2  شماره 

صفحات  -

تاریخ انتشار 2004